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FOREWORD 


The  work  reported  In  this  document  was  performed  at  the  Crew 
Station  Design  Facility  (CSDF) , Crew  Station,  Escape,  and  Human 
Factors  Branch  (ASD/ENECC)  of  the  Aeronautical  Systems  Division, 
Wright-Patterson  Air  Force  Base,  Ohio,  over  the  period  March  1974 
to  May  1975,  under  Job  Order  Number  ASDD-0008,  on  behalf  of  ASD 
Project  2713,  Task  30  Ground  Proximity  Warning  Study. 

The  report  is  a description  of  a study  investigating  the 
feasibility  of  using  a Ground  Proximity  Warning  System  in  high 
performance,  highly  maneuverable  aircraft.  The  study  was 
conducted  for  the  Flight  Instruments  Branch,  Deputy  for  Engineering, 
Wright-Patterson  AFB,  Ohio,  who  funded  the  necessary  development 
work  on  the  CSDF  simulator  and  its  operating  costs  during  the  study. 

The  principal  investigators  were  Squadron  Leader  Christopher 
J.  Hyatt,  RAF,  and  Mrs  Frances  A.  Knless. 

The  ground  proximity  warning  system  simulator  capability  was 
developed  by  George  Sayre  and  Mark  Christensen  of  the  Singer  Company, 
Simulation  Products  Division,  under  contract  F33-615-71-C-1921 . 
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SECTION  I 


INTRODUCTION 


Much  work  has  been  done  over  the  last  four  years  to  develop 
an  inexpensive,  compact,  lightweight  system  which  would  provide 
adequate  warning  of  an  aircraft’s  Inadvertent  controlled  flight 
into  terrain.  A workable  system  now  exists  and  variants  of  it  are 
being  manufactured  by  different  contractors.  The  system  is  being 
fitted  to  all  commercial  passenger  carrying  jet  aircraft  as  a 
result  of  new  Federal  Aviation  Administration  (FAA)  requirements. 

The  USAF  wishes  to  take  advantage  of  any  benefits  the  system  can 
offer  and  installation  in  all  USAF  passenger  carrying  aircraft  is 
being  considered.  The  question  has  naturally  arisen  whether  the 
system  as  designed,  or  with  suitable  modifications,  could  usefully 
be  applied  to  other  types  of  aircraft  such  as  highly  maneuverable 
combat  aircraft.  This  study,  intended  to  provide  a first  look  at 
the  problem,  was  to  determine  whether  further  investigation  would 
be  worthwhile. 

The  study  was  initiated  as  a result  of  discussions  between  the 
Flight  Instruments  Branch  (ASD/ENFI)  and  the  Human  Factors  Branch 
(ASD/ENCTH)  over  the  period  October  73  to  February  74.  It  was 
conceived  as  a low  cost  study,  with  most  of  the  expenditure  directed 
to  the  creation  of  a Ground  Proximity  Warning  System  (GPWS) 
simulation  capability.  The  study  was  carried  out  in  two  phases.  The 
first  consisted  of  flying  a Visual  Flight  Rules  (VFR)  mission  over  a 
mountainous  terrain  model.  Because  visual  cues  tended  to  conflict 
with  the  warning  system,  a second  phase  was  flown  under  Instrument 
Flight  Rules  (IFR)  conditions,  with  terrain  following  radar  cues 
available  to  the  subject. 


The  apparatus  used  for  conducting  this  experiment  was  the  Crew 
Station  Design  Facility's  (CSDF)  FB-111  simulator  at  Wright-Patterson 
Air  Force  Base.  This  simiHator,  driven  by  a CDC  Sigma  5 computer, 
consists  of  modified  FB-111  cockpit  mounted  on  a 5 degree  of  freedom 
Singer  motion  base  (vertical  and  lateral  translation;  pitch,  roll, 
and  yaw  rotation)  . 

The  cockpit  has  a Wide  Angle  Commercial  (WAC)  window  for  its 
visual  system,  driven  by  a monochrome  television  probe  operating  on 
a SMK  23  terrain  model.  For  the  purpose  of  this  experiment  Singer 
Simulation  Products  Division  developed  a probe  altitude  sensor, 
operated  by  tracking  an  obliquely  impinging  laser  beam.  This 
generates  real-time  altitude-above-ground  signals  from  the  terrain 
model  for  the  radar  altimeter  in  the  cockpit.  This  unique  capability 
allows  the  whole  terrain  model  to  be  used  (rather  than  limited  routes 
with  ground  heights  stored  in  memory)  and  also  permits  changes  to  the 
terrain  model  without  the  need  for  any  software  change  to  the  radar 
altitude  system.  The  simulator  also  has  an  inertial  navigation 
system  and  terrain  following  (TF)  radar,  both  of  which  are  coupled 
into  the  autopilot.  Thus,  fully  automatic  TF  cross-country  missions 
can  be  flown.  Alternatively  the  autopilot  can  be  uncoupled,  and  TF 
and  navigation  data  given  to  the  pilot  by  means  of  the  Attitude 
Director  Indicator  (ADI) . Altitude  data  for  the  TF  system  is 
obtained  from  radar  landmass  plates  representing  actual  North 
American  terrain.  This  is  a standard  system,  but  this  was  the  first 
time  it  had  been  activated  on  the  facility's  simulator. 
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1.  GROUND  PROXIMITY  WARNING  SYSTEM  (GPWS) 


r 


The  simulated  GPWS  installation  took  data  from  the  basic  aero 
program  of  the  simulator.  Radar  altitude  data  was  taken  from  the 
probe  altitude  sensor  for  visual  flight  missions  and  from  the  TF 
system  for  instrument  flight  missions.  The  warning  envelopes  used 
are  shown  in  Figures  1 and  2.  It  was  not  considered  feasible  to 
continually  update  these  to  current  FAA  configurations. 

The  GPWS  is  activated  when  the  aircraft  enters  the  appropriate 
warning  envelope  and  continues  to  give  warning  until  the  aircraft 
leaves  the  envelope.  The  following  forms  of  warning  were  provided: 

a.  A swept  tone  (Whoop-whoop!)  fed  to  the  simulator 
interphone  system. 

b.  A voice  saying  "Pull  Up!",  fed  to  the  simulator 
interphone  system. 

c.  An  illuminable  warning  panel  bearing  the  legend  PULL  UP. 

Switching  facilities  were  available  at  the  simulator  console  to 
select  any  or  all  of  these  warnings.  If  the  voice  and  tone  were 
selected  together,  the>  operated  sequentially,  tone  followed  by 
voice  (Whoop-whoop!  Pull  Up!  Wlioop-whoop ! Pull  Up!). 

i \ 

‘ I 


En  Route  Envelope 


B (Sink  rate)  (ft/min) 

Figure  1 Proximity  Warning  Envelope  (Sink  Rate  Modes 


En  Route  Envelope 


Not  in  landing  configuration 


SECTION  III 

STUDY  PROCEDURE,  PHASE  I (VFR) 


The  first  phase  of  the  study  was  carried  out  under  simulated 
Visual  Flight  Rules  (VFR)  conditions. 

1 . SUBJECTS 

Twelve  subjects  took  part  in  the  first  phase  of  the  study; 
eight  were  rated  USAF  line  pilots  and  four  were  Air  National  Guard 
pilots.  Table  1 shows  their  qualifications  and  flying  experience. 
Their  flying  experience  ranged  from  500  to  5400  flight  hours,  with 
a mean  flight  experience  of  2880  hours. 

2 . PROCEDURE 

Each  subject  received  a 15  minute  prebrief.  He  was  told  he  was 
participating  in  a study  to  evaluate  a ground  proximity  warning 
system  (GPWS)  in  a high-performance,  highly  maneuverable  aircraft. 

It  was  explained  that  his  primary  task  was  to  fly  the  simulator,  but 
that  he  would  be  asked  to  comment  on  the  simulation  and  critique  the 
warnings  he  received.  Each  subject  was  also  briefed  on  the  nature  of 
ground  proximity  warning  systems  and  the  type  of  logic  built  into 
them.  The  four  possible  modes  or  situations  which  would  trigger  a 
warning  by  the  system  were  explained  to  him. 

After  the  briefing,  the  subject  was  given  a 30-minute  trial  run 
to  familiarize  him  with  the  FB-111  simulator  and  its  flight 
characteristics.  This  training  ride  consisted  of  completing  several 
360°  turns  at  various  bank  angles,  configuring  the  aircraft  for 
landing,  shooting  touch-and-gos,  varying  the  wing  sweep,  and  changing 
airspeed  and  altitude. 

A flight  planning  exercise  followed  the  training  ride.  This 
consisted  of  studying  the  flight  plan  and  a map  of  the  terrain  over 
which  the  mission  would  be  flown.  The  route  was  pre-marked  on  the 


TABLE  1 

PHASE  I SUBJECT  EXPERIENCE 


jb.iect 

Qualification 

Type  Duty 

Flying 
Time  (hrs) 

1 

Pilot 

ANG* 

900 

2 

Senior  Pilot 

Instructor  Pilot 

4000 

3 

Pilot 

Test  Pilot 

500 

4 

Command  Pilot 

Aircraft  Commander 

4600 

5 

Senior  Pilot 

ANG 

3100 

6 

Senior  Pilot 

Fighter  Pilot 

2800 

7 

Senior  Pilot 

Fighter  Pilot 

2100 

8 

Senior  Pilot 

Fighter  Pilot 

2300 

9 

Senior  Pilot 

Non-Flying  Status 

5400 

10 

Senior  Pilot 

ANG 

3450 

11 

Pilot 

ANG 

1000 

12 

Senior  Pilot 

Fighter  Pilot 

4400 

Average  flying  experience  = 2879.12  hours. 
*ANG  - Air  National  Guard  Fighter  Pilot 


r-  1 

Any  questions  the  subject  had  regarding  the  mission,  or  what  he 
was  expected  to  do,  were  answered  at  this  time. 

: 

3.  MISSION 

The  VFR  mission  flight  plan  is  described  in  Appendix  A.  There 
were  four  locations  along  the  route  where  the  pilot  was  scheduled  to 
encounter  situations  which  would  cause  him  to  receive  one  of  the  four 
modes  of  warning.  These  locations,  however,  were  not  marked  on  the 
subject's  copy  of  the  map.  The  four  situations  were: 

a.  An  airpocket  after  a takeoff 

b.  Closure  into  "mountainous"  terrain 

c.  Descent  over  a flat  desert 

d.  Excessive  descent  rate 

Ideally,  these  four  situations  in  the  mission  placed  the  pilot  in 

i simulated  conditions  where  he  received  each  of  the  four  modes  of  the 

[ 

! GPWS : 

a.  Mode  3 - Altitude  loss  after  climbout  (negative  climb  rate)  ! 

b.  Mode  2 - Excessive  closure  rate  ■ 

c.  Mode  4 - Unsafe  terrain  clearance  based  on  airplane 

s configuration  i 

' d.  Mode  1 - Excessive  descent  rate  (sink  rate)  with  respect  to 

• the  terrain. 

4.  EXPERIMENTAL  DESIGN 

Each  subject  was  given  two  of  the  three  methods  of  warning. 

* He  would  sometimes  receive  them  singly  and  sometimes  simultaneously; 

e.g. , during  different  parts  of  one  mission,  the  pilot  might 
receive  a tone  warning  alone,  a voice  warning  alone,  or  a voice  and 
tone  warning  together  (Whoop-whoop!  Pull  Up!  Whoop-whoop!  Pull  Up!). 


Table  2 shows  the  order  in  which  the  types  of  warning  were 
presented  to  subjects. 

5 . DATA  COLLECTION 

An  8-channel  Brush  recorder  was  used  to  collect  four  aircraft 
parameters : 

1)  Delta  stick  (degrees) 

2)  Delta  pitch  (degrees) 

3)  Delta  flight  path  vector  (degrees) 

4)  Delta  barometric  height  (feet) 

and  four  pilot  response  parameters: 

5)  Plus  and  minus  vertical  velocity  (feet  per  minute) 

6)  Radar  altitude  (feet) 

7)  Indicated  airspeed  (knots) 

8)  Percent  throttle 

The  first  parameters  were  recorded  when  the  pilot  entered  a 
warning  situation.  The  recording  equipment  took  the  instantaneous 
value  of  the  four  aircraft  parameters  at  the  time  when  the  warning 
was  triggered  and  recorded  changes  in  stick  movement,  pitch,  flight 
P^th  vector,  and  barometric  height  throughout  the  duration  of  the 
warning.  These  four  channels  registered  "0"  as  long  as  the  pilot 
received  no  warnings,  and  ten  seconds  after  the  termination  of  a 
warning  they  reset  to  "0"  again.  The  last  four  channels  were 
plotted  continuously  during  the  entire  mission. 


TABLE  2 


SCHEDULE  OF  WARNING  SIGNALS 


Subiect 

Mode  3 

Mode  2 

Mode  4 

Mode  1 

I 

tone 

voice/ tone 

voice/ tone 

tone 

2 

tone 

tone 

voice 

voice 

3 

light 

voice 

voice/light 

light 

4 

tone 

light 

light /tone 

tone 

5 

voice 

voice/light 

light 

voice 

6 

light 

light/ tone 

tone 

light 

7 

voice 

voice/ tone 

tone 

voice 

8 

voice/light 

light 

voice 

voice/light 

9 

light/ tone 

tone 

light 

light/tone 

10 

voice 

VO ice /tone 

voice/ tone 

tone 

11 

voice 

voice/light 

voice/light 

light 

12 

light 

light/ tone 

light/tone 

tone 

In  addition  to  the  eight  analog  channels,  the  Brush  recorder 
had  two  digital  channels.  One  was  used  to  record  elapsed  mission 
time  and  the  other  to  indicate  a warning  condition.  Thus  it  was 
simple  to  determine  the  point  in  time  when  the  warning  started  and 
for  how  long  it  was  activated. 

A magnetic  tape  was  also  used  to  collect  data,  on  the  above 
parameters  and  a number  of  others,  at  a rate  of  10  iterations  per 
second.  The  tape  data  was  collected  as  a backup  to  the  Brush 
recordings . 

6 . QUESTIONNAIRE 

In  order  to  obtain  pilot  reactions  to  the  quality  and  validity 
of  the  different  warnings  received,  a questionnaire  was  developed 
(Figures  3 and  4) . This  questionnaire  was  aimed  at  gaining 
information  about  pilots'  acceptance  of  the  GPWS,  their  reactions  to 
the  different  methods  of  presenting  warning  information,  and  the 
usefulness  of  such  a system  in  the  F-111  or  in  similar  aircraft.  The 
first  part  of  the  questionnaire  was  deliberately  unstructured  in  the 
hope  of  attracting  any  original  ideas  the  subjects  might  have 
concerning  GPWS.  The  second  part  was  structured  to  ensure  that 
certain  basic  information  was  obtained. 
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GROUND  PROXIMITY  WARNING  SYSTEM  STUDY 
TEST  PROFILE 
CRITIQUE-PART  1 


NAME: 

COMMENTS  ON  SIMULATION: 
VISUAL: 


MOTION: 


SYSTEM  RESPONSE /CONTROLS 


COMMENTS  ON  WARNING  SYSTEM: 

TYPE  OF  SITUATION  IN  WHICH  WARNING  OCCURRED: 


MERITS  OF  DIFFERENT  METHODS  OF  PRESENTING  WARNING  INFORMATION: 


Figure  3.  Test  Profile  Critique-Part  1 


GROUND  PROXIMITY  WARNING  STUDY 
TEST  PROFILE  CRITIQUE  PART  2 


1.  Did  you  get  any  warnings  that  were 

a.  Unnecessary? 

b . Too  early? 

c.  Too  late? 


2.  Did  you  meet  situations  where  you  felt  you  should  have  been 
warned  but  were  not? 


3.  During  the  course  of  the  mission  you  received  various  types  of 
warnings  and  combination  of  warnings.  For  each  warning  and 
combination  of  warnings,  describe  what  you  liked  or  did  not  like 
and  for  what  reasons.  Which  warning  did  you  feel  was  most 
effective? 


4.  Did  you  feel  the  system  was  useful  or  could  be  made  useful. 

a.  For  general  flight  in  F-111  or  similar  aircraft. 

b.  For  transit/ferry  flight  in  F-111  or  similar  aircraft. 


Figure  4.  Test  Profile  Critique-Part  2 
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SECTION  IV 

STUDY  PROCEDURE,  PHASE  II  (IFR) 


During  Phase  I of  the  study  it  was  found  that  the  visual  cues 
a pilot  receives,  particularly  if  he  has  no  lateral  vision,  may 
appear  to  him  to  conflict  with  the  GPWS  warnings.  This  problem 
is  inherent  in  the  GPWS,  since  the  radar  altitude  data  is  always 
historical  with  respect  to  the  pilot's  forward  field  of  view. 

Thus,  an  obstacle  which  has  passed  safely  below  the  pilot's  line 
of  sight  may  still  cause  a momentary  warning  signal  as  the  radar 
altimeter  senses  it.  It  was  therefore  decided  to  run  a phase 
without  this  visual  conflict,  by  using  an  IFR  mission.  This  is 
in  line  with  the  expected  use  of  the  GPWS.  To  qJ ve  the  subjects 
some  form  of  reference,  the  TF  capability  of  the  simulator  was 
used,  with  the  pilot  in  the  control  loop,  i.e.,  the  mission  was 
flown  manually  using  the  ADI  display. 

1 . SUBJECTS 

In  view  of  the  high  skill  level  required  to  fly  the  FB-111 
On  manual  TF,  and  the  need  for  familiarity  with  the  TF  to  permit 
assessment  of  the  GPWS  against  it,  three  experienced  F-111  pilots 
were  used  as  subjects  for  this  phase. 

2 . PROCEDURE 

Each  subject  was  given  the  same  prebrief  as  described  in 
Phase  I . He  was  then  briefed  on  the  requirement  for  manual  TF 
flying  and  the  type  of  terrain  he  would  be  flying  over.  He  was 
given  a map  marked  with  the  test  route.  The  coordinates  of  the 
route  were  preset  into  the  automatic  inertial  navigation  system, 
with  system  output  fed  to  the  vertical  bar  of  the  ADI. 
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TF  data  was  fed  to  the  horizontal  bar  of  the  ADI.  Thus,  by 
manually  flying  the  aircraft  in  response  to  the  ADI  command  signals, 
the  correct  route  was  flown  at  the  selected  TF  height.  This  was 
fully  explained  to  each  subject,  and  he  was  advised  of  the  TF  height 
selection  and  ride  setting  (either  hard  or  soft)  he  would  be  using. 

3.  MISSION 

The  IFR  mission  flight  plan  is  described  in  Appendix  B.  The 
mission  was  over  terrain  varying  from  1000  feet  to  6000  feet  MSL 
with  peaks  to  9000  feet.  The  route  was  essentially  along  valleys, 
but  in  view  of  the  speed  (475  knt)  the  contours  were  not  followed 
precisely,  so  there  were  considerable  fluctuations  in  ground  height 
along  track. 

4.  EXPERIMENTAL  DESIGN 

Because  of  limitations  on  time  and  availability  of  subjects  for 
Phase  II,  the  experimental  concept  used  was  to  progressively  push  the 
man  in  the  loop  towards  the  limit  of  the  manual  TF  capability.  The 
missions  flown  by  each  subject  are  detailed  in  Table  3. 

5.  DATA  COLLECTION 

The  same  data  collection  program  as  in  Phase  I was  used, 
including  Brush  recorder,  magnetic  tape,  and  questionnaires.  In 
addition  to  the  above  data,  an  experimenter  recorded  the  time  of 
warning,  mode  of  warning,  and  any  comments  the  subject  made  about 
the  warning.  The  comments  the  subjects  made  as  they  received  each 
warning  during  the  IFR  phase  are  given  in  Appendix  B. 


TABLE  3 


TERRAIN  FOLLOWING  (TF)  MISSION  DATA 


MISSION 

SUBJECT 

HEIGHT  (FT) 

RIDE 

NUMBER 

1 

1 

1000 

Hard 

2 

1000 

Soft 

3 

750 

Soft 

4 

2 

750 

Hard 

5 

750 

Soft 

6 

3 

750 

Hard 

7 

400 

Soft 

NOTES: 

1.  Pitch  was  manually  controlled  throughout.  ■ 

2.  indicated  Airspeed  was  475  knt  throughout.  j 

3.  Mission  one  was  flown  with  automatic  roll  control.  j 

4.  Mission  three  and  all  subsequent  missions  were  flown  with  mode 

1 (excessive  sink  rate)  disabled.  I 

i 


I 

1 


SECTION  V 


j RESULTS 

The  VFR  and  IFR  phases  of  the  experiment  are  independent  and 
can  be  considered  separately.  A considerable  amount  of  hard  data 
was  acquired  for  the  VFR  phase,  whereas  the  IFR  phase  data  was 
mainly  a series  of  subjective  assessments  by  the  subject  pilots. 

1.  PHASE  I REACTION  DATA 

Although  much  hard  data  was  recorded,  it  was  not  susceptible 
to  precise  or  significant  statistical  analysis;  although  the  study 
was  initially  designed  with  adequate  data  points  and  a measure  of 
counter-balancing,  various  practical  problems  during  the  data 
, collection  weakened  the  design.  An  analysis  of  these  problems  is 

presented  in  Appendix  C. 

A wide  range  of  parameters  was  recorded  to  ensure  that  an 
appropriate  measure  could  be  found.  The  data  on  the  Brush  recorder 
i was  found  to  be  sufficiently  comprehensive,  and  the  back  up  data  on 

tape  was  therefore  erased  to  permit  reuse  of  the  tape  and  to  save 
storage  space. 

I 

As  a result  of  inaccurate  route  flying,  not  only  were  some  data 
points  lost,  but  many  warnings  were  received  at  points  where  none 

I 

was  intended,  and  these  were  not  received  consistently  by  all 
subjects.  It  was  therefore  considered  inadvisable  to  use  them  for 
comparative  purposes,  but  the  data  has  been  retained  in  case  a more 
extensive  analysis  seems  desirable  at  a later  date. 

Inspection  of  the  various  parameters  showed  them  to  be 
generally  consistent  but  some  were  affected  by  factors  outside  the 
scope  of  this  study;  any  data  recorded  more  than  4 or  5 seconds 
after  warning  were  affected  by  changes  in  ground  height  at  or  after 
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the  time  of  warning.  It  appeared  that  a good  measure  of  response 
to  the  warnings  was  given  by  the  time  to  achieve  certain  criteria. 

For  Modes  1 and  3,  which  are  triggered  by  negative  barometric  rate, 
return  to  0 barometric  rate  appeared  to  be  a good  criterion;  whereas 
for  Mode  2,  which  is  triggered  by  radar  rate,  a gain  in  barometric 
height  of  150  feet  was  used.  Mode  4 was  not  considered,  due  to 
insufficient  data  (one  data  point).  The  times  taken  to  achieve 
criterion  values  are  shown  in  Table  4 together  with  the  reciprocal 
times  which  represent  rates  of  response.  Since  Mode  3 was  the  only 
mode  for  which  data  was  obtained  from  every  subject,  the  subjects 
have  been  arranged  ordinally  with  respect  to  Mode  3 response  time. 

No  order  is  apparent  in  the  related  Mode  1 and  2 responses,  which 
indicates  that  there  are  no  consistent  between-subject  differences. 

The  mean  response  rate  for  each  warning  mode  is  also  shown  in 
Table  4.  To  provide  some  degree  of  comparability  between  modes 
(since  the  modes  have  arbitrary  and  different  criteria  for 
measurement  of  response  time) , response  rates  within  modes  have  been 
expressed  as  deviations  from  the  mean  response  rate  for  that  mode. 

These  deviations  are  grouped  in  Table  5 by  nature  of  warning 
signal.  Two  general  trends  are  apparent  here: 

a.  Single  warnings  led  to  responses  which,  on  balance,  were 
better  than  the  average  response.  Combined  warnings,  e.g.,  voice 
and  light  together,  led  to  worse  than  average  responses. 

b.  Within  the  single  warnings,  the  response  to  voice  was  much 
faster  than  the  response  to  tone  or  light. 

It  would  be  wrong  to  draw  firm  conclusions  from  this,  since  the 
data  are  not  statistically  significant,  but  there  is  enough  evidence 
to  suggest  that  further  experiments  specifically  designed 
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RESPONSE  RATE  DEVIATIONS  BY  WARNING  SIGNAL 


to  establish  the  relationship  between  type  of  warning  and  speed  of 
response  would  be  worth-while.  There  is  also  the  suggestion  that 
presenting  two  warnings  simultaneously  may  have  an  adverse  effect, 
whether  due  to  some  form  of  mental  conflict  over  interpretation,  or 
perhaps  some  anxiety  associated  with  the  intensifying  effect  of  the 
double  warning. 

2.  PHASE  I QUESTIONNAIRE 

Some  comments  from  the  questionnaire  are  given  at  Appendix  D. 
Some  general  conclusions  from  these  questionnaires  are  that; 

a.  Only  one  subject  felt  that  the  system  failed  to  warn  him 
of  a dangerous  situation. 

b.  Only  two  subjects  felt  that  some  of  the  warnings  were  too 

late . 

c.  Most  subjects  felt  that  some  of  the  warnings  were  given 
too  early,  or  were  even,  in  some  cases,  unnecessary. 

d.  Only  one  subject  felt  that  the  system  could  not  usefully 
be  applied  to  high  performance  aircraft. 

e.  The  subject's  ratings  of  the  various  types  of  warning  were 
imprecise  verbal  descriptions.  On  two  separate  occasions  the 
experimenters  independently  transcribed  these  ratings  onto  a 1-5 
scale  (5=most  favorable) . These  four  transcriptions  proved  to  be 
highly  consistent  both  between  experimenters  and  over  a period  of 
time.  Average  values  are  shown  in  Table  6,  together  with  overall 
averages  by  warning  signal.  The  subject's  assessments  are  slightly 
in  conflict  with  the  reaction  times  in  Table  4 and  5. 
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3.  PHASE  II  REACTION  DATA 

It  was  quickly  discovered  in  the  first  two  tests  flown  under 
IFR  conditions  that  the  Mode  1 (rate  of  descent)  warning,  as 
currently  configured,  was  useless  for  a terrain-following  flight 


TABLE  6 


SUBJECT  ASSESSMENT  OF  WARNING  SIGNALS 


SUBJECT 

NUMBER 

VOICE 

TONE 

LIGHT 

COMBINATION 

1 

4.50 

2 

3.75 

4.25 

3 

4.00 

4.00 

5.00 

4 

4.00 

1.75 

3.50 

5 

4.50 

2.25 

6 

3.25 

4.25 

4.50 

7 

5.00 

4.00 

8 

4.00 

5 .00 

4.75 

9 

4.50 

2.75 

4.50 

10 

2.00 

2.75 

4.00 

11 

4.75 

3.00 

4.75 

12 

4.00 

1.00 

4.25 

AVERAGE 

4.00 

3.50 

3.43 

4.63 

(Numerical  values  assigned  by  experimenters  on  the  basis  of  subjects' 
verbal  ratings.  Scale  1-5,  5 most  favorable.) 


path.  Given  a good  terrain  following  signal,  it  is  quite  safe  to 
climb  and  descend  at  rates  well  into  the  GPWS  Mode  1 envelope.  On 
the  other  hand,  because  of  the  tendency  to  maintain  constant 
terrain  clearance,  the  Mode  2 envelope  remained  relatively  usable 
much  lower  altitudes  than  might  have  been  expected.  Once  the 
Mode  1 had  been  disabled,  the  Mode  2 warnings  were  found  to  be 
mostly  appropriate.  However,  the  subjects  felt  that,  assessed 
against  the  available  TF  cues,  the  GPWS  warnings  were  slightly 
later  than  they  would  have  liked. 

These  results  are  not  an  indictment  of  the  GPWS;  they  simply 
indicate  that  it  is  not  as  flexible,  precise,  or  responsive  as  a 
higher  grade  system.  On  the  other  hand  neither  is  it  so  heavy  or 
so  costly,  and  it  was  never  intended  to  cope  with  the  type  of 
environment  used  in  this  study.  Unfortunately,  this  adverse 
comparison  with  the  TF  system  (which,  being  fitted  to  all  F-llls, 
provided  an  easy  base  reference)  led  all  the  experienced  F-111 
pilots  to  deprecate  the  GPWS  because  it  would  not  result  in  any 
improvement  to  the  F-111  as  they  knew  it. 

4.  PHASE  II  QUESTIONNAIRE 

Some  comments  from  the  questionnaires  are  given  in  Appendix  E. 
An  examination  of  the  subjective  comments  given  by  the  subjects  at 
the  time  of  each  warning  shows  that  the  terrain  following  radar 
(TFR)  system  is  felt  to  be  superior  to  the  ground  proximity  warning 
system.  Many  comments  pointed  out  that  the  ground  proximity  system 
could  only  look  at  the  terrain  directly  beneath  the  aircraft  while 
the  TFR  could  look  forward  at  upcoming  terrain.  In  the  majority  of 
the  cases  the  two  systems  working  together  led  to  confusion  for  the 
pilots.  The  TFR  pitch  command  sometimes  gave  the  pilot  a pitch  down 
command  at  the  same  time  as  the  ground  proximity  system  was  telling 
the  pilot  to  "PULL  UP!". 
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SECTION  VI 
CONCLUSIONS 


Although  in  many  ways  the  study  appears  to  have  been 
inconclusive,  in  terms  of  hard  data  and  significant  results,  in 
fact  it  has  led  to  a considerably  better  understanding  of  the 
implications  of  the  GPWS. 


It  is  clear  that,  with  the  exception  of  the  negative  response 
exhibited  by  TF  experienced  pilots,  there  was  a high  degree  of 
pilot  acceptance.  This  would  presumably  be  reinforced  by  a suitably 
developed  GPWS  which  produced  more  consistently  acceptable  warnings. 


Some  preconceived  ideas  were  upset  regarding  the  relative 
suitability  of  GPWS  for  different  types  of  aircraft  and  mission. 

There  appears  to  be  some  degree  of  trade-off  between  the  ease  with 
which  a high  performance  aircraft  can  get  into  trouble  and  its 
ability  to  get  out.  With  the  exception  of  the  terrain  following 
situation  it  appeared  likely  that  a workable  series  of  envelopes 
could  be  developed  for  combat  aircraft.  Further  studies,  preferably 
in  flight,  should  be  made.  On  the  other  hand,  parallel  studies  with 
the  GPWS  in-flight  on  a KC-135  suggest  there  could  be  a greater 
problem  with  transport  aircraft  than  was  expected  because  of  higher 
load  factors  on  military  aircraft  and  the  need  for  tactical  flight 
profiles.  The  inability  of  the  system  to  cope  with  terrain  following 
is  not  a serious  shortcoming  since  the  whole  concept  of  Ground 
Proximity  Warning  is  predicated  on  "inadvertent  flight  into  terrain" 
which  is,  by  implication,  from  a nominally  safe  height. 

There  appears  to  be  some  evidence  that  the  voice  warning  has 
benefits  despite  the  fact  that  it  was  not  particularly  favored  by 
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the  subjects.  This  is  certainly  worth  pursuing  in  a dedicated 
Study.  CSDF  initially  had  some  reservations  about  the  use  of  the 
warning  "Pull  Up",  but  after  some  analysis  it  was  retained  for  the 
study  and  is  currently  favored  by  CSDF.  The  rationale  for  this, 
briefly,  is  that: 

a.  The  system  can  only  work  if  nuisance  warnings  are  avoided. 

b.  This  can  only  be  achieved  by  keeping  the  envelopes  small. 

c.  GPWS  is  not  a routine  flight  aid,  but  an  emergency  warning 
for  an  aircraft  which  has  inadvertently  entered  a dangerous  flight 
condition . 

d.  Warning  will  always  be  at  the  last  minute. 

e.  Instant  response,  without  the  delay  which  would  be  caused 
by  assessing  possible  failures,  is  needed. 

f.  The  warning  "Pull  Up”,  while  not  universally  appropriate, 
Offers  the  chance  of  saving  more  aircraft  than  a more  complex  or 
more  abstract  warning. 

As  stated  above,  however,  this  is  an  analytical  judgement  and  we 
know  of  no  data  on  alternative  voice  messages  for  GPWS.  Work  in 
this  field  is  currently  being  undertaken  at  NASA,  Ames  Research 
Center,  and  close  cooperation  for  the  future  is  strongly 
recommended . 


APPENDIX  A 
VFR  MISSION  PROFILE 

The  VFR  mission  was  flown  over  the  visual  terrain  model  of  the 
F-111  simulator.  This  Is  a SMK  23  model  which  was  extensively 
remodeled  for  this  study.  Figure  A-1  is  a map  of  the  model,  showing 
part  of  the  mission  route.  Table  A-1  shows  the  flight  plan  which 
was  given  to  the  subjects. 
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FLIGHT  PLAN  FOR  PHASE  I MISSION 
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APPENDIX  B 


IFR  MISSION  PROFILE 

The  IFR  mission  was  flown  over  a radar  simulation  of 
southeastern  Arizona,  using  a radar  landmass  plate.  The  route 
followed  is  shown  on  the  map  in  Figure  B-1 . The  route  plan  was 
programmed  into  the  automatic  navigation  system  so  there  was  no 
need  to  give  the  subjects  a flight  plan. 


• ~ f i .... . ....  — i 

J 


TPC  G-19D-S 


SCALE  1:500,000 


I 


APPENDIX  C 


PROBLEMS  AFFECTING  EXPERIMENTAL  DESIGN 

There  were  three  basic  problems  which  interfered  with  the 
experimental  design,  and  made  it  impossible  to  apply  any  rigorous 
statistical  analysis  to  the  data.  One  was  the  difficulty  in 
obtaining  suitable  subjects.  More  important  was  the  problem  of 
achieving  consistency  in  navigation.  Finally,  there  were  some 
short -comings  in  simulation  fidelity. 

AVAILABILITY  OF  Sl'B.JECTS 

The  experiment  was  essentially  a low  budget  project  with  all 
available  funds  being  applied  to  the  development  of  the  necessary 
hardware  for  radar  altimeter  simulation.  It  was  also  an  internal 
..  ASD- project  so  that  there  was  no  user  command  sponsorship.  As  a 

result,  all  the  subjects  were  on-base  volunteers  from 
Wrigh t-Patterson  AFB,  who  had  to  be  scheduled  at  times  convenient  to 
them  and  to  their  commanders.  Their  schedules  also  had  to  be  closely 
coordinated  with  the  schedule  for  the  FB-111  simulator,  which  shares 
its  computer  with  other  simulator  facilities.  Specific  effects  of 
these  factors  were: 

a.  There  was  no  spare  pool  of  subjects  to  permit  subject 
selection  or  flexible  scheduling. 

b.  Most  subjects  were  not  familar  with  the  F-111,  and  those 
who  were  familiar  were  not  current. 

c.  Time  was  not  available  for  extra  training  or  for  reruns. 

The  net  result  of  these  problems  was  that  the  skill  levels  of  the 
subjects  differed  widely,  and  the  average  level  was  lower  than  could 
have  been  wished  for.  Any  problems  in  administering  or  recording 
the  test  runs  had  to  be  accepted  since  there  was  no  time  available 
for  reruns. 
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NAVIGATION  PROBLEMS 


I The  available  terrain  model  for  flying  the  tests  represents 

I very  rugged  country.  Although  this  was  good  in  the  way  it 

simulated  a high  risk  area  where  GPWS  might  be  important,  it  had 

I 

! the  major  drawback  that  very  small  changes  in  height,  position,  or 

I heading  generated  major  changes  in  the  aspect  of  the  terrain 

• presented  to  the  system  and  the  pilot.  When  this  is  considered  in 

I conjunction  with  the  low  skill  level  mentioned  in  the  previous 

paragraph,  it  will  be  seen  that  there  was  little  chance  of 
presenting  consistent  situations  to  the  subjects.  The  test  routes 
were  chosen  to  produce  four  specific  warnings.  In  practice  the 
four  warnings  were  generated  under  widely  differing  situations  from 
test  to  test,  or  in  some  cases  the  system  did  not  trigger  a warning 
at  all.  In  particular.  Mode  3 was  generally  swamped  by  Mode  1 
i signals,  and  Mode  4 was  only  achieved  by  one  subject.  On  the  other 

^ hand,  many  warnings  were  generated  at  points  where  none  were 

: Intended;  the  pilot  was  therefore  specifically  instructed,  as  he 

i 

! approached  some  of  these  situations,  to  ignore  the  inadvertent 

warning.  Although  this  procedure  was  undesirable,  since  it  probably 

i 

j had  some  interaction  with  his  reaction  to  other  warnings,  it  appeared 

to  be  the  most  workable  and  expedient  solution  to  the  problem. 

These  inconsistencies  in  the  conditions  under  which  the  four 
planned  warnings  were  generated  had  the  following  effects: 

a.  The  visual  cues,  and  the  timing  of  the  warning  relative  to 
these  cues,  were  not  constant. 

The  aircraft  flight  vector  and  position  were  not  constant, 
c.  The  Impending  crisis  to  be  recovered  from  was  not  constant. 
Any  attempt  at  precise  comparisons  would  therefore  be  suspect. 
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SIMULATION  FIDELITY 


Two  areas  of  simulation  had  insufficient  fidelity  to  guarantee 
meaningful  results.  The  radar  altimeter  simulation,  which  was  a 
state-of-the-art  facility  specifically  produced  for  this  study,  was 
still  liable  to  produce  occasional  erratic  outputs  as  a result  of 
its  sensitive  nature  and  the  heavy  filtering  used  to  compensate  for 
this.  Thus,  it  is  possible  that  not  all  warnings  were  generated  at 
precisely  the  correct  time  to  simulate  a real  GPWS.  No  quantitative 
Assessment  of  this  problem  was  attempted. 

The  second  problem  was  in  the  visual  cues  available  to  the 
pilot,  which  were  restricted  by  the  45°  (diagonal)  field  of  view  of 
the  simulator.  Mountain  peaks  were  perceived  by  the  subjects  to 
disappear  below  the  field  of  view,  and  mentally  dismissed  as  already 
passed.  From  the  aircraft  safety  point  of  view  this  dismissal  is 
correct  as  the  pilot  can  correctly  assess  he  will  not  hit  the  peak. 
However,  the  GPWS,  with  no  forward  looking  facility,  will 
subsequently  "see"  the  peak  below,  and  give  warning-  If  the  peak  is 
500  feet  below,  and  the  ground  speed  is  400  knt  the  warning  may  occur 
up  to  3 seconds  after  the  peak  disappears;  at  lower  speeds  the  time 
will  be  commensurate ly  longer.  To  the  pilot,  who  has  now  ceased  to 
consider  the  peak  as  a threat,  this  rates  as  a "nuisance"  warning. 

The  15°  downward  field  of  view  is  realistic,  but  more  visual  cues 
out  to  the  side  would  help  hiui  to  assess  the  time  of  passing  over  an 
object . 

CONCLUSION 

The  combined  effect  of  these  various  problems  caused  the  loss  of 
some  data  points  and  made  others  suspect  in  varying  degrees.  It  was 
therefore  not  considered  legitimate  to  carry  out  a formal  statistical 
analysis,  but  a genera]  trend  assessment  was  made. 
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APPENDIX  D 

PILOTS  COMMENTS  ON  VFR  MISSION 


I.  COMMENTS  ON  AND  SUGGESTIONS  FOR  WARNING  SYSTEM: 

A.  PRO 

"Warning  system  was  O.K.  but  it  would  be  helpful  to  know 
what  approximate  limits  are  required  to  set  it  off  and  safely 
'escape ' . " 

"...  Would  like  an  indication  of  the  type  of  warning 
indicated,  i.e.  too  low,  sink  rate,  etc." 

"...  Would  possibly  like  to  see  three  separate  forms  of 
warning,  e.g.,  voice:  PULL  UP/LEVEL  OFF/GO  ROUND." 

"...  The  too  rapid  descent  signal  should  be  differentiated 
from  the  too  low  signal." 

"The  system  could  be  made  useful  particularly  in  weather 
or  instrument  approaches  to  strange  fields." 

"System  could  be  useful  as  a warning  system  in  the  F-111 
or  similar  aircraft,  but  not  as  means  of  conducting  low-level 
terrain  avoidance  flight." 

"The  most  useful  time  for  the  warnings  seems  to  be  during 
take-off  and  landing,  but  during  high-speed  phases  I have  doubts 
about  its  usefulness  because  it  does  not  look  ahead  of  the 
aircraft . " 

B.  CON 

"There  were  several  warnings  that  occurred  when  I felt 
the  aircraft  was  well  within  the  envelope  of  safe  clearance 
considering  the  maneuverability  of  that  type  of  aircraft." 

"A  system  that  cries  'wolf  too  often  will  be  ignored;  and 
the  system  seemed  to  give  warnings  too  often,  especially  during 
landing  approaches  as  compared  with  visual  information." 

"...  maybe  a longer  time  delay  between  an  out-of-envelope 
condition  and  a warning  signal  would  reduce  the  warnings  caused 
by  momentary  pitch  inputs." 

"...  landing  warnings  in  VMC  are  not  necessary-coulc' 
distract  during  visual  approaches  or  during  VMC  flight." 
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"This  system  is  a duplication  of  the  TFR  and  can't  be  | 

used  for  TFR  flight  missions."  j 

"It  could  be  of  some  use;  iiowever,  existing  warning 
systems  tend  to  make  the  addition  of  this  cost-prohibitive." 

II.  SUMMARY  OF  SUBJECTS  WHO  FELT  THEY  RECEIVED  WARNINGS  THAT  WERE: 

unnecessary- 11  out  of  12  subjects,  in  situations  such 
as  low  altitude  approach,  slow  descent  over  the  desert  and  clearing 
some  terrain. 

too  early-  6 out  of  12  pilots  said  "yes"  for  reasons  such 

as  during  rapid  rates  of  descent  and  flying  over  some  terrain. 

too  late-  only  one  subject  thought  a warning  was  too  late 

"....  I believe  it  was  when  we  approached  a mountain  top  and  an 

excessive  rate  of  pitch  up  was  required  to  stop  the  (voice)  warning." 

III.  COMMENTS  RECEIVED  ON  THE  THREE  TYPES  OF  WARNING  (LIGHT,  TONE, 

VOICE)  AND  COMBINATION  OF  THE  WARNINGS: 

LIGHT 

A . PRO 

"Prefer  the  visual  warning." 

"I  like  light  better  than  the  tone,  it  was  more  effective." 

"The  light  warning  is  in  a good  location." 

"...  light  better  cue  than  audio  due  to  faster  response  i 

especially  during  IMC  to  a visual  cue Light  alone  may  be  | 

missed.  i 

Light  location  good-in  F-111  the  location  could  be  confused  with  TF 
fail  light,  but  both  require  the  same  response.  For  a single 
warning  I feel  light  is  more  effective  due  to  my  tendency  to  respond 
to  visual  cues  more  rapidly  than  audio  (ones)." 

"Did  not  encounter  light,  but  feel  it  would  be  more 
distinctively  identifiable  than  aural  signals.  ...  visual  signal 
such  as  in  simulation  is  much  preferred  in  my  opinion  because  it  is 
immediately  identified  as  a terrain  avoidance  signal  so  pilot's 
response  is  probably  much  quicker." 

B.  C(7N 

"I  did  not  respond  to  tlie  light  at  the  time  even  though  I 
consciously  saw  it." 


"...  red  lights  in  cockpit  scare  fighter  pilots.  Blinking 
or  steady  red  lights  in  cockpit  cause  distraction  when  attention 
should  be  outside  the  cockpit." 

"Warning  light  not  always  noticed  when  it  came  on  while 
flying  visually." 

TONE 


A.  PRO 

"Liked  this  best,  however  type  of  tone  should  be  changed. 
Sounds  too  much  like  an  emergency  beeper." 

"I  liked  tlie  tone  ...  should  be  loud  enough  to  "wake  you  up. 

"Tone  warning  seemed  to  get  my  attention  quicker  at  first... 

B.  CON 

"I  think  the  tone  should  be  louder  and  used  in  conjunction 
with  the  light." 

"Aural  signals  in  general  can  easily  be  masked  out  by 
intercom,  UHF  communication,  RHAWS  warnings,  gear  warning,  aircraft 
noise,  etc.  (Particularly  in  a combat  environment.) 

"...  could  be  easily  confused  with  the  emergency  beeper  on 
personal  locator  beacon." 

VOICE 


A.  PRO 

"Voice  is  better  than  light  expecially  in  landing  phase." 

"Liked  this  signal  the  best  because  it  is  enough  different 
from  all  other  aircraft  warnings  to  get  your  attention  quickly. 
Unecessary  warnings  could  defeat  this  advantage  though." 

"...  always  noticed  - good  attention  getter  ...  regardless 
of  circumstances." 

B.  CON 

"...  change  the  voice  that  recorded  it!  It  sounded  too  much 
like  a robot  and  it  got  on  my  nerves." 

"Voice  not  preferred  due  to  large  number  of  other  UHF 
communications  in  a combat  environment." 


"...  did  not  trigger  a response  at  first  because  of 
similarity  to  radio/interphone  transmissions.  A female  voice  or  a 
combination  of  tone/voice  might  have  been  better." 

Combinations 

"Both  warnings  (light  and  voice)  together  are  effective." 

"Tone  and  light  good  for  last  chance  warning,  i.e.,  tone 
only  in  caution  area,  then  light  also  when  terrain  gets  very  close." 

"The  aural  tone  and  light  combination  are  good  (built-in 
backup  and  calibration)." 

"Ought  to  have  both  audio  and  visual  "PULL  UP"  warning  come 
on  at  the  same  time." 


APPENDIX  E 

PILOTS  COMMENTS  ON  IFR  MISSION 


The  following  are  conunents  made  by  the  pilots  during  the  actual 
running  of  the  IFR  portion  of  the  simulation.  The  subjects  were 
instructed  to  comment  on  each  warning  as  they  received  it,  i.e.,  was 
it  valid,  a nuisance,  inconsistent  with  the  TFR  pitch  command. 

"...  That  was  very  confusing,  ground  proximity  is  telling 
me  to  pull  up  and  an  opposing  point  of  view  from  the  TFR  pitch 
command  telling  me  to  pitch  down." 

"Two  conflicting  points  of  view  and  a person  will  tend  to 
ignore  one  or  the  other." 

"...  change  parameters  so  a warning  won't  come  on  as  often." 

"TFR  doesn't  give  you  any  warning  because  it  looks  ahead 
and  knows  the  glitches  in  the  radar  altimeter  are  only  temporary." 

"No  indication  of  radar  altimeter  dropping  but  we  are 
getting  warnings." 

"Warning  ground  proximity  is  giving  me  information  that  TFR 
is  not,  but  need  training  to  see  what  the  warning  means." 

"these  (warnings)  are  all  ridiculous  since  I'm  getting  the 
pitch  down  command  on  the  TFR." 

"I  was  in  an  80“  climb  and  I still  got  a warning." 

"it  (warning)  turns  on  at  500  feet  and  TFR  is  set  to  fly 
at  400  feet." 

"...  ignore  the  system,  the  GPWS  is  annoying." 

"Warnings  are  very  distracting  with  TFR." 

"Sink  rate  warning  is  coming  on.  TFR  ignores  sink  rate 
directly.  It  is  radar  rate  for  TFR.  False  warnings  should  be 
eliminated  with  TFR  set  on  soft  ride." 

"no  good  reason  for  that  one." 

"yes,  we  were  closing  then." 

"going  around  corners  seems  to  trigger  a warning." 
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